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A method of image registration and medical image data processing apparatus 



The present invention relates to image registration, and more particularly 
without limitation, to image registration in medical diagnostic systems. 

In current clinical practice, it is not uncommon that several medical images are 
to be compared or combined. For example, this occurs when images of a single individual are 
acquired by different techniques, each producing different information about the individual. 
One technique may depict brain function very well and another a brain tumour. 

The combination of the two is crucial in determining whether the tumour can 
be resected without damaging vital brain areas. A second example is that of images of an 
individual taken over a period of time. This can be used, among other things, to study the 
development of anatomical structures. Besides the comparison of images of a single 
individual, comparisons of images of different individuals is also performed, e.g. to study the 
differences and similarities between groups. 

To be able to compare images, the contents of the images must be in 
alignment. This is often not the case for numerous reasons, such as movement of the 
individual or differences in the field of view (the part of the anatomy that is captured in the 
images). The process of finding the correspondence between the contents of images is called 
image registration. This is achieved by optimizing a measure of similarity between images, 
such as the position of corresponding landmarks or the similarity in intensity of 
corresponding anatomical structures. 

In most cases, global translations and rotations suffice to align images. When 
one image has to be deformed, however, to fit the contents of the other this is referred to as 
non-rigid or elastic registration. This is necessary for registration of images of different 
individuals (who have different anatomies) or images of a single individual when an 
anatomical change has taken place. 



Various image registration techniques are known from the prior art, such as 
from WO 01/ 01346 Al, EP 1 184 802 A2, US 2002 / 0 141 626 Al , US 6,266,453 Bl . 
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The book "Medical Image Registration" by Josef Hajnal provides an overview 
of the various prior art image registration techniques. As far as elastic image registration is 
concerned the article "Elastic Image Registration and Pathology Detection" by Paul 
Thompson and Arthur W. Toga 

(http://ww.lom.ucla.edu/~mompson^DF/no^ics_ElasChptpdf) provides an overview of 
the known approaches for elastic image registration. This article is also published as a book 
chapter in "Handbook of Medical Image Processing", Academic Press, 2000. 

A common disadvantage of prior art image registration methods is the lack of 
an objective quality criterion. So far, the only reliable way to check the results of the image 
registration is visual assessment by a human observer. This is a practical problem that 
substantially limits the application of registration methods. For example, in case of time 
series where many sequential images have to be registered it is impractical to interrupt an 
otherwise automatic pre-processing method. This is also true if registration is part of a pre- 
processing chain. 



It is therefore an object of the present invention to provide for an improved 
method of image registration which addresses the problems of the prior art. It is a further 
object of the present invention to provide for an improved computer program product and 
improved medical image data processing apparatus for image registration. 

The present invention provides for an improved method of image registration 
where registrations are performed for a number of sub-volumes of the image to be registered 
in addition to a global registration. For example, a number of sub-volumes contained in the 
image data is selected manually by an operator by marking the sub-volumes of interest. 

For this purpose a graphical user interface (GUI) can be utilized. Alternatively 
or in addition an automatic image segmentation can be performed for identification of such 
sub-volumes. For example, each of the sub-volumes contains a certain medical region of 
interest, such as an organ or a tumour. 

In accordance with the present invention a registration is performed for each 
one of the sub-volumes in addition to a global registration of the image data. This enables to 
identify an outlier sub-volume which substantially deviates from the transformation of the 
other sub-volumes and the global registration. Such an outlier sub-volume can be cut off in 
order to improve the overall registration result or it can trigger a subsequent elastic 
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registration for improved registration. The transformation parameter set obtained in the first 
registration step can be used as an input parameter set for the subsequent elastic registration. 

The present invention is particularly advantageous as it registers sub-volumes 
of the images and translates the registration results to global transformations. In particular 
this enables a comparison of the resulting transformations which indicates: 

the accuracy of the global transformation, e.g. by measuring variations, 
the reason for the inaccuracy, e.g. too much constraint transformation, 
the location of the problem in the images, e.g. local image artefacts. 
In some applications outliers in the registration of the sub-volumes are likely 
to originate from image artefacts or unequal anatomies in the images such as pathology, 
especially if the structures can be assumed as rigid, such as the scull. An example is neural- 
MR where a sub-volume is selected on a tumour that does not exist on the other image. 
Another example is neural-CT where a sub-volume is selected on image artefacts caused by 
metal fillings. A possible reaction to this detected origin of inaccuracy can be to signal for 
user-interaction or to cut-off this section for a more accurate global registration. 

If the image is to be registered e.g. in the abdomen with the deforming organs 
such as the liver an additional processing step is performed for improved registration rather 
than cutting-off the corresponding sub-volume. An inaccurately registered sub-volume can be 
caused by deformation of the organ, e.g. due to breathing motion, which cannot be 
appropriately dealt with by a rigid, global transformation. In this case a refinement and / or 
localization of the transformation is performed in order to improve the registration of this 
sub-volume. For this improved registration the transformation can be locally extended with 
more degrees of freedom; in particular non-rigid or elastic image registration can be used for 
this purpose. 

Another advantageous application of the present invention is the detection of 
artefacts in MR inter-leaved scans, i.e. the even and odd slices being not aligned. Image 
interpolation between the slices results in double edges and other artefacts which make 
registration inaccurate and can result in a low quality after the reformatting which is 
subsequent to the registration. The present invention enables to detect such artefacts in order 
to warn a user that artefacts are present in the MR inter-leaved scans. 

In essence, the invention provides for a sub-regional registration to be 
performed alongside the global registration of the full images. Sub-regions or sub-volumes 
of the images to be registered are selected and are registered throughout the sequence of 
images. In accordance with a preferred embodiment a table of registration information is 
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compiled for all sub-regions and the global registration containing translation and rotation 
information. Quantification of the registration process is therefore possible over sub-regions 
and allows identification of inaccuracies within the overall registration process. 

In particular, this enables to draw the users attention to the sub-volume or sub- 
volumes of the global image where the registration process has failed and manual orientation 
is reqmred. Alternatively the result of the registration method of the invention can be 
incorporated into a fully automated image handling sequence which enables spurious sections 
of the sequence of images to be removed. 



In the following preferred embodiments of the invention are explained in 
greater detail by making reference to the drawings in which: 

Fig. 1 is illustrative of aflow diagram of an embodiment ofamethod of the 



invention, 



system. 



Fig. 2 is a schematic example for the registration of sub-volumes, 

Fig. 3 is a block diagram of a medical image data acquisition and processing 



Fig. 1 shows a flow diagram of an embodiment of the registration method of 
the mvention. la step 100 image data is provided by an imaging modality. Depending on the 
application all of the image data are provided by the same image modality. Alternatively the 
image data of different image modalities are to be combined. For example, X-ray imaging 
magnetic resonance imaging (MRI), computer tomography (CI) imaging, functional MM 
(fMRD, single photon emission computer tomography (SPECT) or positron emission 
tomography (PET) are used as image modalities. 

In step 102 a selection of sub-volumes ft in the image data is performed This 
can be done manually by marking the sub-volumes ft on a display or by performing an image 
segmentation step or another suitable image processing step for identification of sub-regions 
or sub-volumes in the image data. 

In step 104 image registrations are performed locally for the selected sub- 
volumes ft. For example, these registrations are performed by means of a rigid image 
registration method. In step 106 a parameter set T, is outputted for each one of the sub- 
volumes ft for which a registration has been performed. Each one of the transformation 
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parameter sets T- is descriptive of the displacement of the corresponding sub-volume S,- in the 
two images to be registered. 

In step 108 a global registration is performed for the images to be registered. 
Again a rigid image registration method can be employed for this purpose. In step 1 10 a 
global transformation parameter set T G is outputted. The global transformation parameter set 
To is descriptive of the displacement of the images to be registered with respect to each other. 

In step 1 12 the index i is initialized to be equal to one. 

In step 1 14 a mean distance measure is calculated for the transformation 
parameter set T,. This can be done by calculating a distance measure between the 
transformation parameter set T, and each one of the other transformation parameter sets Tj 
and to the global transformation parameter set T G and calculating a mean value on the basis 
of these individual distance measures. For this purpose any distance measure for the 
determination of the distance between two parameter sets or vectors can be used, such as 
Euclid's distance. For example Euclid's distance is calculated for all pairs of parameter sets 
T , T jp where i * j. and the resulting Euclid's distances are added. This provides the mean 
distance measure. 

In step 1 16 it is determined whether the mean distance measure of step 1 14 is 
above a threshold level. If this is not the case the registration of the corresponding sub- 
volume Si is deemed to be sufficiently accurate and the control goes to step 120 to increment 
the index i. 

If the mean distance measure is above the threshold the control goes from step 
1 16 to step 1 1 8. In step 1 1 8 an output signal is generated for identification of the sub-volume 
Si. This output signal can be outputted as a warning signal by means of a graphical user 
interface in order to warn an operator that the registration of the sub-volume % is inaccurate. 
Alternatively or in addition the output signal generated in step 1 1 8 triggers the execution of 
an additional non-rigid, elastic image registration procedure for improved registration of the 
sub-volume S, and / or for improved global registration. 

As an alternative to steps 1 1 4 and 1 1 6 another statistical method can be used 
to identify an outlier parameter set and corresponding sub-volume. 

Fig. 2 shows a schematic example. The example is illustrative of an 
application in two dimensions for convenience of explanation. It is to be noted that the 
example can be extended to more dimensions. 
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Image 100 is a slice or "slab" of image data which has been acquired by 
medical image data acquisition. Image 100 is located in the XY-plane 104. Image 102 is also 
within plane 104 and needs to be registered with image 100. 

Image 100 contains the sub-volumes A, B, C and D. These sub-volumes are 
selected by marking the corresponding regions in image 1 00 by means of a graphical user 
interface by an operator and / or by a segmentation step or another suitable data processing 
step for identification of such regions of interest, linage 102 has corresponding sub-volumes 
A',B', C'andD'. 

An image registration is performed both globally for images 100 and 102 and 
for the individual sub-volumes. This means that the sub-volumes A and A' are registered as 
well as the sub-volumes B-B', C-C and D-D*. 

Table 106 of Fig. 2 shows the resulting transformation parameter sets. In the 
example considered here the transformation parameter set for each individual image 
registration contains the amount of displacement in the X and Y directions within plane 104 
as well as the amount of angular transformation in degrees of rotation. 

From table 106 it is apparent that the transformation parameter set for the 
registration of the sub-volume B-B' substantially deviates from the other transformation 
parameter sets. The sub-volume B-B' is therefore identified as an outlier. In response a 
corresponding warning signal is outputted and / or an elastic registration procedure is 
triggered for improved registration of the sub-volume B-B'. The fact that the transformation 
parameter set of the sub-volume B-B' is an outlier can be determined by processing table 106 
by means of suitable statistical methods and / or by calculating of distance measures. 

Fig. 3 shows a block diagram of a computer 200 which is coupled to a medical 
image data acquisition apparatus 202. Computer 200 has memory 204 for the storage of 
image data and has a graphical user interface 206. Further computer 200 has segmentation ' 
program module 208, image registration program module 210, elastic image registration 
program module 212 and memory 214 for the storage of transformation parameters (cf. table 
106 of Fig. 2). In addition computer 200 has program 216 for controlling the operation of 
these various components. Computer 200 is coupled to display 218 for rendering of image 
data. Alternatively or in addition computer 200 is connected to a printer. 

In operation the image data is loaded into memory 204 from image data 
acquisition apparatus 202. The image data is rendered on display 21 8. By means of graphical 
user interface 206 an operator can select a number of sub-volumes in the image data 
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Alternatively or in addition sub-volumes are selected automatically by means of 
segmentation program module 208. 

Image registrations are performed by means of program module 210 for 
registering of the sub-volumes and for performing a global image registration. The 
corresponding transformation parameters are stored in memory 214 for analysis by program 
216. When program 216 identifies an outlier sub-volume a warning message is generated and 
outputted on display 218. Alternatively or in addition program 216 triggers program module 
212 for performing an elastic image registration for the outlier sub-volume. 



WO 2004/063990 

LIST OF REFERENCE NUMBERS: 



PCT/IB2003/005898 



5 



10 



100 


XXA1C1KG 


102 


• 

xLUctgC 


104 


piane 


106 

X \J\J 


UIOJ.O 


200 


i^oinpuier 


Z\)Z 


image data acquisition apparatus 


204 


memory 


206 


GUI 


208 


segmentation program module 


210 


program module 


212 


program module 


214 


memory 


216 


program 


218 


display 



